• 专利附录
  • 专利说明
  • 权利要求
  • 类似技术
  • 同族专利
  • 引用文献
  • 法律状态
  • 优先权
    • 专利摘要:
    The present invention relates to a coaching binocular stereo vision device and a method for acquiring high-precision stereo vision images, which belong to the field of stereo vision technologies and solve a problem that existing stereo vision devices cannot meet 5 requirements for high precision and low cost at the same time. The device includes: a binocular stereo vision unit used for off-line collecting binocular sample images within a field of view and on-line collecting binocular real-time images within a measured field and used for processing and obtaining binocular visible non-stereo images, matching cost diagrams and binocular stereo vision images; a coaching unit used for acquiring 10 precise stereo vision images within the same field of view, and an intelligent learning unit used for training a deep convolutional neural network stored in the intelligent learning unit until the deep convolutional neural network converges according to one binocular visible non-stereo image of the binocular visible non-stereo images, the matching cost diagrams and the binocular stereo vision images obtained based on the 15 binocular sample images and according to the precise stereo vision images, and also used for obtaining high-precision stereo vision images within the measured field based on corresponding images obtained by processing the binocular real-time images after convergence.
    公开号:NL2026030A
    申请号:NL2026030
    申请日:2020-07-09
    公开日:2021-02-09
    发明作者:Shi Congling；He Jingze；Che Honglei；Xu Yuanfei
    申请人:Beijing Hangxing Machine Mfg Co Ltd；China Academy Safety Science & Technology；
    IPC主号:
    专利说明:

    -1- A Coaching Binocular stereo Vision Device and a Method for Acquiring High- precision stereo Vision Images Technical Field The present invention relates to a technical filed of correction of binocular vision images, in particular to a coaching binocular stereo vision device and a method for acquiring high-precision stereo vision images. Background Binocular stereo vision is an important form of machine vision, which is a method for acquiring two images of a measured object from different positions with an imaging apparatus based on a principle of parallax, and acquiring three-dimensional geometric information of the object by calculating the position deviation between corresponding points of the images. A depth measurement method based on binocular stereo vision is similar to that of human eyes. Unlike a depth camera based on principles of TOF and structured light, it does not actively project a light source to the outside but completely relies on the two captured pictures (color RGB or grayscale) to calculate a depth. Therefore, it is sometimes called a passive binocular depth camera.
    In practical applications, binocular stereo vision has the following disadvantages: 1) very sensitive to ambient illumination. The binocular stereo vision method relies on natural light in the environment to collect images, and due to the influence of environmental factors such as changes in illumination angle and illumination intensity, brightness difference between the two captured pictures will be large, which will pose a great challenge to a matching algorithm; 2) not suitable for monotonous scenes that lack texture. Since the binocular stereo vision method performs image matching based on visual features, 1t will be difficult to match scenes lacking visual features (such as the sky, white walls, deserts, etc.), resulting in large matching errors and even matching failures.
    A laser TOF stereo vision apparatus can effectively solve the problems of the binocular stereo vision method and has high measurement precision. However, due to its high apparatus cost, the laser TOF stereo vision apparatus is severely limited to be extensively
    22. used. How to achieve high-precision and low-cost stereo vision is an urgent problem to be solved. Summary In light of the above analysis, the present invention is intended to provide a coaching binocular stereo vision device and a method for acquiring high-precision stereo vision images to solve the problem that existing stereo vision devices cannot meet the requirements for high precision and low cost.
    The purpose of the present invention is mainly achieved by the following technical solutions: A coaching binocular stereo vision device includes: a binocular stereo vision unit used for off-line collecting binocular sample images within a field of view and on-line collecting binocular real-time images within a measured field; and used for processing and obtaining binocular visible non-stereo images, matching cost diagrams and binocular stereo vision 1mages based on the binocular sample images or the binocular real-time images; a coaching unit used for acquiring precise stereo vision images within the same field of view as the binocular sample images; an intelligent learning unit used for training a deep convolutional neural network stored in the intelligent learning unit until the deep convolutional neural network converges according to one binocular visible non-stereo image of the binocular visible non-stereo images, the matching cost diagrams and the binocular stereo vision images obtained based on the binocular sample images; and also used for obtaining high-precision stereo vision images within the measured field according to one binocular visible non-stereo image of the binocular visible non-stereo images, the matching cost diagrams and the binocular stereo vision images obtained based on the binocular real-time images after the training is completed.
    Based on the above solution, the present invention has also made the following improvements: further, the binocular stereo vision unit includes: two visible cameras with the same parameters and one controller;
    -3- the controller is used to control the two visible cameras to collect images at the same time and process the two images collected at the same time to obtain the binocular visible non-stereo images, the matching cost diagrams and the binocular stereo vision images. Further, the coaching unit includes an infrared laser emitter and a high frame frequency infrared camera, and 1s used to output the precise stereo vision images based on an infrared laser TOF principle. Further, before training the deep convolutional neural network, the intelligent learning unit also corrects the binocular visible non-stereo images, the matching cost diagrams and the binocular stereo vision images obtained based on the binocular sample images and corrects the precise stereo vision images according to transformation relationship between coordinates of the binocular stereo vision unit and the coaching unit, and unifies them into the same coordinate system; the transformation relationship is determined by: aligning the binocular stereo vision unit with the coaching unit, establishing a coordinate system by using a calibration object within the same field of view as a reference, and determining the transformation relationship between the coordinates of the binocular stereo vision unit and the coaching unit.
    Further, the intelligent learning unit trains the deep convolutional neural network by performing the following operations: training the deep convolutional neural network by using the one binocular visible non- stereo image of the binocular visible non-stereo images, the matching cost diagrams and the binocular stereo vision images after correction as three channel-alignment inputs and using the precise stereo vision images of the coaching unit after correction as positive sample images; when deviation between stereo vision images output by the trained deep convolutional neural network and the positive sample images satisfies a threshold condition, obtaining a converged deep convolutional neural network; otherwise, modifying each pixel of the positive sample images according to the following formula:
    -4- CT kl + hol + Rn Ty TOT fy > Ta wherein Ts) represents a pixel value of a pixel with coordiantes (x, y ) in the stereo vision images output by the trained deep convolutional neural network; Li represents a pixel value of a pixel with coordiantes (x, y ) in the precise stereo vision images of the coaching unit after correction; Diy represents a pixel value of a pixel with coordiantes (x, y) in the binocular stereo vision images after correction; i. and Lo represent a maximum pixel value and a minimum pixel value of a pixel with coordinates (x, y ) in the stereo vision images output by the trained deep convolutional neural network, respectively; and k, k, and k, are weight coefficients corresponding to Ton Lies and Don, respectively; continuing to train the deep convolutional neural network by using the modified positive sample images; if the deep convolutional neural network converges, obtaining a converged deep convolutional neural network; otherwise, repeating the above modification process.
    Further, the controller processes the two images within the measured field collected at the same time by using an SGB algorithm or a BM algorithm to obtain the binocular visible non-stereo images, the matching cost diagrams and the binocular stereo vision images.
    The present invention also discloses a method for acquiring high-precision stereo vision images, wherein the method includes the following steps: on-line collecting binocular real-time images within a measured field, and processing and obtaining binocular visible non-stereo images, matching cost diagrams and binocular stereo vision images based on the binocular real-time images; inputting one binocular visible non-stereo image of the binocular visible non-stereo images, the matching cost diagrams and the binocular stereo vision images obtained based on the binocular real-time images to a trained deep convolutional neural network,
    -5- and using the deep convolutional neural network to process and output high-precision stereo vision images.
    Based on the above solution, the present invention has also made the following improvements: further, the method also includes training the deep convolutional neural network by: off-line collecting binocular sample images within a field of view, and processing and obtaining binocular visible non-stereo images, matching cost diagrams and binocular stereo vision images based on the binocular sample images; acquiring precise stereo vision images within the same field of view as the binocular sample images at the same time; correcting the binocular visible non-stereo images, matching cost diagrams and binocular stereo vision images processed and obtained based on the binocular sample images, and correcting the precise stereo vision images; training the deep convolutional neural network by using the one binocular visible non- stereo image of the binocular visible non-stereo images, the matching cost diagrams and the binocular stereo vision images after correction as three channel-alignment inputs of the deep convolutional neural network and using the precise stereo vision images after correction as positive sample images; when deviation between stereo vision images output by the trained deep convolutional neural network and the positive sample images satisfies a threshold condition, obtaining a converged deep convolutional neural network; otherwise, modifying each pixel of the positive sample images according to the following formula: M= [ Ts) Taw < Ty < Taas 6 Vl + el + Dj Te < Tor Tj > Th, Wherein Les) represents a pixel value of a pixel with coordiantes (x, y ) in the stereo vision images output by the trained deep convolutional neural network; Lies represents a pixel value of a pixel with coordiantes (x, y ) in the precise stereo vision images after correction; Di represents a pixel value of a pixel with coordiantes (x, y ) in the
    -6- binocular stereo vision images after correction; T va and Ly represent a maximum pixel value and a minimum pixel value of a pixel with coordinates (x, y ) in the stereo vision images output by the trained deep convolutional neural network, respectively; and k, , k, and k, are weight coefficients corresponding to Tis , Lis) and Des ‚ respectively; continuing to train the deep convolutional neural network by using the modified positive sample images; if the deep convolutional neural network converges, obtaining a converged deep convolutional neural network; otherwise, repeating the above modification process.
    Further, the correcting the binocular visible non-stereo images, matching cost diagrams and binocular stereo vision images processed and obtained based on the binocular sample images, and correcting the precise stereo vision images includes: establishing a coordinate system by using a calibration object within the same field of view as a reference, and determining transformation relationship between the coordinates of the binocular stereo vision unit and the coaching unit; correcting the binocular visible non-stereo images, the matching cost diagrams and the binocular stereo vision images obtained based on the binocular sample images and correcting the precise stereo vision images according to the transformation relationship, and unifying them into the same coordinate system. Further, when a position, the field of view or an ambient light source for acquiring the binocular real-time images are significantly changed, retraining the deep convolutional neural network.
    The beneficial effects of the present invention are as follows: the operation of the coaching binocular stereo vision device provided by the present invention is specifically divided into an off-line training process and an on-line use process. During off-line training, binocular vision images with relatively low precision are obtained by using a binocular stereo vision unit with low cost, precise stereo vision images are obtained by using a high-precision coaching unit; a trained deep convolutional neural network is obtained by using the deep convolutional neural
    -7- network to determine their relationship; the coaching unit can be removed after the training, and high-precision stereo vision images are obtained merely by cooperation of the binocular stereo vision unit and an intelligent learning unit, thereby obtaining high- precision stereo vision images by using binocular vision components with low cost and low precision. This method is suitable for places where installation position of the binocular stereo vision unit is relatively fixed and an ambient light source changes little, such as subway stations. The method of the present invention is implemented based on the same principle with the above device and thus also has the effects that the above device can achieve.
    In the present invention, the above technical solutions can also be combined with each other to achieve many more preferred combination solutions. Other features and advantages of the present invention will be described subsequently in the description, and some advantages may become apparent from the description or be understood by implementing the present invention. The objects and other advantages of the present invention can be realized and obtained through the contents particularly pointed out in the description, claims and drawings. Brief Description of the Drawings The drawings are only for the purpose of illustrating specific embodiments, and are not considered to limit the present invention. Throughout the drawings, the same reference indicates the same component. Fig. 1 is a schematic structural diagram of a coaching binocular stereo vision device during off-line training according to an embodiment of the present invention; Fig. 2 is a schematic structural diagram of a coaching binocular stereo vision device during on-line use according to an embodiment of the present invention; Fig. 3 is a flowchart of a method for acquiring high-precision stereo images according to an embodiment of the present invention.
    Detailed Description Preferred embodiments of the present invention are described in detail below with reference to the accompanying drawings, wherein the drawings constitute a part of the present application and are used to explain the principles of the present invention
    -8- together with the embodiments of the present invention and are not intended to limit the scope of the present invention. Embodiment 1 A specific embodiment of the present invention discloses a coaching binocular stereo vision device, wherein the device includes: a binocular stereo vision unit used for off- line collecting binocular sample images within a field of view and on-line collecting binocular real-time images within a measured field; and used for processing and obtaining binocular visible non-stereo images, matching cost diagrams and binocular stereo vision images based on the binocular sample images or the binocular real-time images, a coaching unit used for acquiring precise stereo vision images within the same field of view as the binocular sample images; an intelligent learning unit used for training a deep convolutional neural network stored in the intelligent learning unit until the deep convolutional neural network converges according to one binocular visible non-stereo image of the binocular visible non-stereo images, the matching cost diagrams and the binocular stereo vision images obtained based on the binocular sample images and according to the precise stereo vision images output by the coaching unit; and also used for obtaining high-precision stereo vision images within the measured field according to one binocular visible non-stereo image of the binocular visible non-stereo images, the matching cost diagrams and the binocular stereo vision images obtained based on the binocular real-time images after the training is completed. These high-precision vision images have characteristics of high resolution and high definition. After learning by the neural network, the binocular stereo vision device can effectively distinguish passenger flows and obtain improved precision of images and better performance before learning.
    Wherein Fig. 1 is a schematic structural diagram of a coaching binocular stereo vision device during off-line training according to an embodiment of the present invention; and Fig. 2 is a schematic structural diagram of a coaching binocular stereo vision device during on-line use according to an embodiment of the present invention.
    Compared with the prior art, the operation of the coaching binocular stereo vision device provided by the present invention is specifically divided into an off-line training process and an on-line use process. During off-line training, binocular vision images with relatively low precision are obtained by using a binocular stereo vision unit with low
    -9- cost, precise stereo vision images are obtained by using a high-precision coaching unit; a trained deep convolutional neural network is obtained by using the deep convolutional neural network to determine their relationship; the coaching unit can be removed after the training, and high-precision stereo vision images are obtained merely by cooperation of the binocular stereo vision unit and an intelligent learning unit, thereby obtaining high-precision stereo vision images by using binocular vision components with low cost and low precision. This device is suitable for places where installation position of the binocular stereo vision unit is relatively fixed and an ambient light source changes little, such as subway stations.
    Preferably, this embodiment also provides a typical method for setting the binocular stereo vision unit, wherein the unit includes: two visible cameras with the same parameters at a certain distance and one controller; wherein the controller is used to control the two visible cameras to collect images at the same time and process the two images within the measured field collected at the same time by using an SGB algorithm or a BM algorithm to obtain the binocular visible non-stereo images, the matching cost diagrams and the binocular stereo vision images; and may also temporarily store the above three types of image information; preferably, in order to obtain precise stereo vision images, the coaching unit of this embodiment includes an infrared laser emitter and a high frame frequency infrared camera, and is used to output the precise stereo vision images based on an infrared laser TOF principle by using the high precision of the hardware itself. Since the positions of the images captured by the binocular stereo vision unit and the coaching unit may deviate, in order to ensure more precise processing results, the intelligent learning unit also needs to correct the binocular visible non-stereo images, the matching cost diagrams and the binocular stereo vision images obtained based on the binocular sample images and correct the precise stereo vision images according to transformation relationship between coordinates of the binocular stereo vision unit and the coaching unit, and unifies them into the same coordinate system before training the deep convolutional neural network; this embodiment provides a method for determining the transformation relationship: aligning the binocular stereo vision unit with the coaching unit, establishing a coordinate system by using a calibration object within the
    -10 - same field of view as a reference, and determining the transformation relationship between the coordinates of the binocular stereo vision unit and the coaching unit.
    The deep convolutional neural network can be trained once images are corrected and unified into the coordinate system. In this embodiment, the intelligent learning unit trains the deep convolutional neural network by performing the following operations: training the deep convolutional neural network by using the one binocular visible non- stereo image of the binocular visible non-stereo images, the matching cost diagrams and the binocular stereo vision images after correction as three channel-alignment inputs and using the precise stereo vision images of the coaching unit after correction as positive sample images; When deviation between stereo vision images output by the trained deep convolutional neural network and the positive sample images satisfies a threshold condition (the threshold is a measure of difficulty of generating positive excitation by a neuron, in memory learning, the threshold is generally set by a toolbox function that comes with it, which is generally a default value and may also be debugged based on customer requirements, determining the length of training time and training effects), obtaining a converged deep convolutional neural network; otherwise, modifying each pixel of the positive sample images according to the following formula: p=! fi Tan < To < Ta | (KT) + Heks + KD) Ty < Bat Tp 5} > Toa wherein Ties represents a pixel value of a pixel with coordiantes (x, y ) in the stereo vision images output by the trained deep convolutional neural network; Li) represents a pixel value of a pixel with coordiantes (x, y ) in the precise stereo vision images of the coaching unit after correction; Dos) represents a pixel value of a pixel with coordiantes (x, y ) in the binocular stereo vision images after correction, 1 and To represent a maximum pixel value and a minimum pixel value of a pixel with coordinates (x, y ) in the stereo vision images output by the trained deep convolutional neural network, respectively, which may be determined by: sorting pixel values of pixels within
    -11 - a certain range around the coordinates (x, y ). the maximum value in the sorting results is used as Lo and the minimum value is used as Lo. k, k, and k, are weight coefficients corresponding to Tie Lis and Dos, respectively; continuing to train the deep convolutional neural network by using the modified positive sample images, the method by which the positive sample images are modified enables the learning unit to keep continual memory learning; if the deep convolutional neural network converges, obtaining a converged deep convolutional neural network; otherwise, repeating the above modification process. This modification method enables it to modify general images of the binocular unit. Once the threshold condition of modification is satisfied later, 1t means that modification ability of the method can enable the binocular unit and the learning unit to generate high-precision stereo vision images similar to that of the coaching unit. This method is a basic learning method for neural network memory learning. (The weight coefficients of the system are set by a toolbox function that comes with the program, which is generally a default value and may also be debugged based on customer requirements, determining the length of training time and training effects) Embodiment 2 Another embodiment of the present invention also discloses a method for acquiring high- precision stereo vision images, as shown in Fig. 3, the method includes the following steps: Step SI: on-line collecting binocular real-time images within a measured field, and processing and obtaining binocular visible non-stereo images, matching cost diagrams and binocular stereo vision images based on the binocular real-time images; Step S2: inputting one binocular visible non-stereo image of the binocular visible non- stereo images, the matching cost diagrams and the binocular stereo vision images obtained based on the binocular real-time images to a trained deep convolutional neural network, and using the deep convolutional neural network to process and output high- precision stereo vision images.
    In specific implementation process, training the deep convolutional neural network by:
    -12- Step S201: off-line collecting binocular sample images within a field of view, and processing and obtaining binocular visible non-stereo images, matching cost diagrams and binocular stereo vision images based on the binocular sample images; Step S202: acquiring precise stereo vision images within the same field of view as the binocular sample images at the same time; Step S203: establishing a coordinate system by using a calibration object within the same field of view as a reference, and determining the transformation relationship between the coordinates of the binocular stereo vision unit and the coaching unit; Step S204: correcting the binocular visible non-stereo images, the matching cost diagrams and the binocular stereo vision images obtained based on the binocular sample images and correcting the precise stereo vision images according to the transformation relationship, and unifying them into the same coordinate system; Step S205: training the deep convolutional neural network by using the one binocular visible non-stereo image of the binocular visible non-stereo images, the matching cost diagrams and the binocular stereo vision images after correction as three channel- alignment inputs of the deep convolutional neural network and using the precise stereo vision images after correction as positive sample images; Step S2051: when deviation between stereo vision images output by the trained deep convolutional neural network and the positive sample images satisfies a threshold condition, obtaining a converged deep convolutional neural network and terminating the training; otherwise, step S2052 is performed, Step S2052: modifying each pixel of the positive sample images according to the following formula: PUT, + Rly + KD TS Dor Ty > Ty Wherein Tis represents a pixel value of a pixel with coordiantes (x, y ) in the stereo vision images output by the trained deep convolutional neural network; Lie represents a pixel value of a pixel with coordiantes (x, y ) in the precise stereo vision images after correction; Dy.) represents a pixel value of a pixel with coordiantes (x, y ) in the binocular stereo vision images after correction; Tv and Ly, represent a maximum
    -13 - pixel value and a minimum pixel value of a pixel with coordinates (x, y ) in the stereo vision images output by the trained deep convolutional neural network, respectively; and k, k, and k, are weight coefficients corresponding to Ties Li and Don, respectively; Step S2053: continuing to train the deep convolutional neural network by using the modified positive sample images (i.e, skip to step S2051); if the deep convolutional neural network converges, obtaining a converged deep convolutional neural network; otherwise, repeating the above modification process.
    A deep convolutional neural network can be obtained through the above process. After the training is completed, inputting one binocular visible non-stereo image of the binocular visible non-stereo images, the matching cost diagrams and the binocular stereo vision images obtained based on the binocular real-time images to a trained deep convolutional neural network, and using the deep convolutional neural network to process and output high-precision stereo vision images. However, when a position, the field of view or an ambient light source for acquiring the binocular real-time images are significantly changed, the trained deep convolutional neural network is no longer applicable, and it is necessary to re-collect images and retrain the deep convolutional neural network based on the changed environmental information. The above method embodiments and device embodiments are based on the same principle, and their relevant points can be used for mutual reference and the same technical effects can be obtained.
    It should be understood by those skilled 1n the art that all or part of the process of implementing the method in the above embodiments may be performed by a computer program instructing relevant hardwares, and the program may be stored in a computer- readable storage medium. Wherein, the computer-readable storage medium may be a magnetic disk, an optical disk, a read-only storage memory, a random storage memory or the like.
    -14 - The above are only preferred specific embodiments of the present invention.
    However, the scope of protection of the present invention is not limited to this.
    Any variation or replacement that is easily conceivable for a person skilled in the art within the technical scope revealed by the present invention should be encompassed within the scope of protection of the present invention.
    权利要求:
    Claims (7)
    [1]
    A coaching binocular stereo vision device, comprising: a binocular stereo vision unit, which is used to collect binocular sample images offline within a field of view and to collect binocular real-time images online within a measured field; and used to process and obtain binocular visible non-stereo images, matching cost diagrams and binocular stereo vision images based on the binocular sample images or the binocular real-time images; a coaching unit, which is used to obtain accurate stereo vision images within the same field of view as the binocular sample images; an intelligent learning unit, which is used to train a deep convolutional neural network, which is stored in the intelligent learning unit until the deep convolutional neural network converges according to one binocular visible non-stereo image from the binocular visible non-stereo images, the corresponding cost diagrams and the binocular stereo vision images obtained based on the binocular sample images and according to the precise stereo vision images output by the coaching unit; and also used to obtain very accurate stereo vision images within the measured field according to one binocular visible non-stereo image of the binocular visible non-stereo images, the matching cost diagrams and the binocular stereo vision images, which are obtained from the binocular real-time images after training is completed; wherein, before training the deep convolutional neural network, the intelligent learning unit also performs correction on the binocular visible non-stereo images, the matching cost diagrams and the binocular stereo vision images obtained from the binocular sample images; and corrects the precise stereo vision images according to the transform relationship between coordinates of the binocular stereo vision unit and the coaching unit; and they unite in the same coordinate system; the transformation relationship is determined by: aligning the binocular stereo vision unit with the coaching unit; setting up a coordinate system using a calibration object inside
    -16 - same field of view as reference, and determining the transformation relationship between the coordinates of the binocular stereo vision unit and the coaching unit. wherein the intelligent learning unit trains the deep convolutional neural network by performing the following operations:
    training the deep convolutional neural network using the one binocular visible non-stereo image of the binocular visible non-stereo images, the matching cost diagrams and the binocular stereo vision images after correction as three channel alignment inputs; and using the accurate stereo vision images of the coaching unit after correction as positive sample images;
    obtaining, when deviation between stereo vision images output from the trained deep convolutional neural network and the positive sample images meets a threshold value, of a converged deep convolutional neural network;
    otherwise modify each pixel of the positive sample images according to the following formula: 7! - Ts) Ls <ls) <Tacx 7) bl, 5) + kyly, + kD, Is) <73.01 Ir)> Tas CT, | »
    where “7 represents a pixel value of a pixel with coordinates (> Y) in the stereo vision images output by the
    Co | Lo, | trained deep convolutional neural network; + "A pixel value of a pixel with coordinates is represented (> Y) in the accurate stereo vision images
    | | The | of the coaching unit after correction; "" "A pixel value of a pixel with Mee. X. . . ne coordinates is represented (‚y) in the binocular stereo vision images after IT T.. we correction; »* And" »a maximum pixel value and a minimum pixel value of a pixel with coordinates is represented (>) in the stereo vision images, which are respectively output by the trained deep convolutional neural k kk | TL, D. network t, 2 and 3 corresponding weight coefficients ~ 0), Cen>), respectively;
    -17 - continuing to train the deep convolutional neural network using the modified positive sample images; as the deep convolutional neural network converges, obtaining a converged deep convolutional neural network; otherwise repeat the above change process.
    [2]
    The coaching binocular stereo vision device according to claim 1, wherein the binocular stereo vision unit consists of: two visible cameras with the same parameters and one controller; the controller is used to control the two visible cameras to collect images simultaneously; and process the two images collected simultaneously to obtain the binocular visible non-stereo images, the matching cost diagrams, and the binocular stereo vision images.
    [3]
    The coaching binocular stereo vision device according to claim 2, wherein the coaching unit has an infrared laser emitter and a high frame rate infrared camera, and this is used to output the accurate stereo vision images based on an infrared laser TOF (Time-of-Day). flight = flight time) principle.
    [4]
    The coaching binocular stereo vision device according to claim 2, wherein the controller processes the two images within the measured field collected at the same time using an SGB algorithm or a BM algorithm to convert the binocular visible non-stereo images, the associated cost diagrams. and obtain the binocular stereo vision images.
    [5]
    A method of obtaining highly accurate stereo vision images, the method comprising the steps of: collecting on-line binocular real-time images within a measured field; and processing and obtaining binocular visible non-stereo images, matching cost diagrams and binocular stereo vision images based on the binocular real-time images; inputting one binocular visible non-stereo image from the binocular visible non-stereo images, the matching cost diagrams and the binocular stereo vision images obtained from the binocular real-time images for
    -18 - a trained deep convolutional neural network; and use the deep convolutional neural network to process and output highly accurate stereo vision images; the method further comprising training the deep convolutional neural network by: collecting on-line binocular real-time images within a measured field; and processing and obtaining binocular visible non-stereo images, matching cost diagrams and binocular stereo vision images based on the binocular real-time images; obtaining accurate stereo vision images within the same field of view and simultaneously from the binocular sample images; correcting the binocular visible non-stereo images, matching cost diagrams and binocular stereo vision images processed and obtained based on the binocular sample images, and correcting the precise stereo vision images; training the deep convolutional neural network using the one binocular visible non-stereo image of the binocular visible non-stereo images, the matching cost diagrams and the binocular stereo vision images after correction as three channel alignment inputs; and using the accurate stereo vision images of the coaching unit after correction as positive sample images; obtaining, when deviation between stereo vision images output from the trained deep convolutional neural network and the positive sample images meets a threshold value, of a converged deep convolutional neural network; otherwise changing each pixel of the positive sample images according to the following formula: T =] be en = KT + Ly + TS Teor Ty> Tg where To is a pixel value of a pixel with coordinates represented (x, y) in the stereo vision images performed by the trained deep convolutional neural network; Lies a pixel value of a pixel
    -19 - is represented by coordinates (x, y) in the accurate stereo vision images of the coaching unit after correction; That) a pixel value of a pixel with coordinates is represented (x, y) in the binocular stereo vision images after correction; Lj and Zo a maximum pixel value and a minimum pixel value of a pixel with coordinates is represented (x, y) in the stereo vision images, which are respectively output from the trained deep convolutional neural network k, k, and k, corresponding weight coefficients Tie, Les and Don, respectively; continuing to train the deep convolutional neural network using the modified positive sample images; as the deep convolutional neural network converges, obtaining a converged deep convolutional neural network; otherwise repeat the above change process.
    [6]
    The method of obtaining high precision stereo vision images according to claim 5, wherein correcting the binocular visible non-stereo images, matching cost diagrams and binocular stereo vision images processed and obtained from the binocular sample images, and correcting the precise stereo vision images. , includes: setting up a coordinate system using a calibration object within the same field of view as a reference; and determining the transformation relationship between the coordinates of the binocular stereo vision unit and the coaching unit, correcting the binocular visible non-stereo images, the matching cost diagrams and the binocular stereo vision images obtained from the binocular sample images; and correcting the precise stereo vision images according to the transformation relationship, and uniting them in the same coordinate system; wherein the binocular stereo vision unit is used to collect binocular sample images offline within a field of view and to collect binocular real-time images online within a measured field; and where the
    -20 - coaching unit is used to obtain the accurate stereo vision images within the same field of view as the binocular sample image.
    [7]
    The method of obtaining accurate stereo vision images according to claim 5, wherein when a position, the field of view or an ambient light source for acquiring the binocular real-time images is significantly changed, retraining the deep convolutional neural network.
    类似技术:
    公开号 | 公开日 | 专利标题
    CN105758426A|2016-07-13|Combined calibration method for multiple sensors of mobile robot
    US7659921B2|2010-02-09|Distance measurement apparatus, distance measurement method, and distance measurement program
    CN100442141C|2008-12-10|Image projection method and device
    JP2016516977A|2016-06-09|Generating a 3D model of the environment
    CN108234984A|2018-06-29|Binocular depth camera system and depth image generation method
    Semeniuta2016|Analysis of camera calibration with respect to measurement accuracy
    Gallego et al.2015|Event-based camera pose tracking using a generative event model
    NL2026030B1|2021-09-23|A Coaching Binocular stereo Vision Device and a Method for Acquiring High-precision stereo Vision Images
    JPH1183530A|1999-03-26|Optical flow detector for image and self-position recognizing system for mobile body
    CN110456330B|2021-07-09|Method and system for automatically calibrating external parameter without target between camera and laser radar
    JP6370478B2|2018-08-08|Optical tracking system and method of calculating posture of marker part of optical tracking system
    CN110189382A|2019-08-30|A kind of more binocular cameras movement scaling method based on no zone of mutual visibility domain
    Solak et al.2015|Distance estimation using stereo vision for indoor mobile robot applications
    Basso et al.2014|Online calibration for networks of cameras and depth sensors
    Wang et al.2020|Soic: Semantic online initialization and calibration for lidar and camera
    CN109993793B|2021-09-07|Visual positioning method and device
    KR20100062320A|2010-06-10|Generating method of robot motion data using image data and generating apparatus using the same
    CN109807937A|2019-05-28|A kind of Robotic Hand-Eye Calibration method based on natural scene
    CN113160327A|2021-07-23|Method and system for realizing point cloud completion
    CN111998862A|2020-11-27|Dense binocular SLAM method based on BNN
    CN111583342A|2020-08-25|Target rapid positioning method and device based on binocular vision
    Su et al.2013|Fast detection method of checkerboard corners based on the combination of template matching and Harris Operator
    Lynch et al.1992|Backpropagation neural network for stereoscopic vision calibration
    JP2018116147A|2018-07-26|Map creation device, map creation method and map creation computer program
    Kheng et al.2010|Stereo vision with 3D coordinates for robot arm application guide
    同族专利:
    公开号 | 公开日
    NL2026030B1|2021-09-23|
    CN110400333B|2020-06-26|
    CN110400333A|2019-11-01|
    引用文献:
    公开号 | 申请日 | 公开日 | 申请人 | 专利标题
    US20190042860A1|2017-08-04|2019-02-07|Samsung Electronics Co., Ltd.|Method and apparatus of detecting object of interest|
    CN106156748B|2016-07-22|2019-03-29|浙江零跑科技有限公司|Traffic scene participant's recognition methods based on vehicle-mounted binocular camera|
    US10803581B2|2017-11-06|2020-10-13|Beijing Keya Medical Technology Co., Ltd.|System and method for generating and editing diagnosis reports based on medical images|
    CN109407697A|2018-09-20|2019-03-01|北京机械设备研究所|A kind of unmanned plane pursuit movement goal systems and method based on binocular distance measurement|CN111260711B|2020-01-10|2021-08-10|大连理工大学|Parallax estimation method for weakly supervised trusted cost propagation|
    法律状态:
    优先权:
    申请号 | 申请日 | 专利标题
    CN201910680675.8A|CN110400333B|2019-07-26|2019-07-26|Training binocular stereoscopic vision device and high-precision stereoscopic vision image acquisition method|
    [返回顶部]